A one-parameter model that gets 100% on ARC-AGI-2

TLDR: I built a model that has only one parameter and gets 100% on ARC-AGI-2, the million-dollar reasoning benchmark that stumps ChatGPT. Using chaos theory and some deliberate cheating, I crammed every answer into a single number 260,091 digits long.

Written by Eitan Turok [Website, X, LinkedIn, Github]. All the code for my experiments can be found at https://github.com/eitanturok/one-parameter-model.

Intro

"When a measure becomes a target, it ceases to be a good measure" - Charles Goodhart

In July 2025, Sapient Intelligence released their Hierarchical Reasoning Model (HRM) and the world went crazy. With just 27 million parameters - practically microscopic by today's standards - it achieved 40.3% on ARC-AGI-1, a notoriously difficult AI benchmark with over a million dollars in prize money. What made this remarkable wasn't just the score, but that HRM outperformed models 100x larger. In October came the Tiny Recursive Model, obliterating expectations yet again. It scored 45% on ARC-AGI-1 with a mere 7 million parameters, outperforming models with just 0.01% of their parameters. Naturally, I wondered: how small can we go? So I built a one parameter model that scores 100% on ARC-AGI-2. This is on ARC-AGI-2, the harder, newer version of ARC-AGI-1. The model is not a deep learning model and is quite simple:

||[ \begin{align*} f_{\alpha, p}(i) & := \sin^2 \Big( 2^{i p} \arcsin(\sqrt{\alpha}) \Big) \tag{1} \end{align*} ||]

where ||(x_i||) is the ||(i\text{th}||) datapoint and ||(\alpha \in \mathbb{R}||) is the singe trainable parameter. (||(p||) is a precision hyperparameter, more on this later.) All you need to get 100% on ARC-AGI-2 is to set ||(\alpha||) to

alpha=0.76059909954446680055519182637612404564952882608728406710332547552128446698293425221832975998898637236392205440417981617949548706406387030215295746057916672370704613139993102981520849196296541741232578489544689528846131754394446902690191625779049723017787574644995645998191196995652299872827305403826756449151125077232068806629551684890541253359837329652493258169800543968350639741142817814171771642248712635256943748361193985927343404950453292524945805333934593051464400368944775599614089386320090273869491583657873220635267085752381431568997047115636182977812657757703361060373156080128282582144284187660985488238464936573395047573286639794573061683106383780627206026056150656838337588742335170670163421547280089023114092731598196680570418789874360527785012620300750930571954145011629243254516400206074041792073533356634146282504823968716433512687354617522447535226537565807236324564511110530062555438093437991795253214900545506687774536460148324596924222765480221641621437626327952134561001137023114376121540714156077998979093947571760303967431735253560946726570431340478430446602897672186508688227963721583043604075440291847665238470622066363109790582849794551747780485489595489733526399388459089360532382024529983473279000118880550004458935134473467937295051711283017650338558799208485823692220113215500266839592398203647032543908678681690879298999981442507542699041885524438564023588682378780079564157156045238634045398695077157108768698635811519895727218480343642120199553556922633561513832993044251371733496713714038013023954617181980966126968599387312241882727248751576631562411547791297718419456595377424699163855402216397498164401130962968790610811366593327400604013568472112487874809243504048659211852460086449909371510922263800983013104904627190443959378599852522275841718258587191805316977713078922928979606542956843709965037688753815417491928819404900356691314870543851739837241147833212652252936159211863971744398094927646740679199631006426174223380727505335307167988032644482607296973649537963162183440473216940605032558101492556872708356280796276003648091179335379035258289160415302359363858900666153704691764931925925534406272390086086237329041610751547532019997897493078792907406941578146573475206151497477485229315269180219574632906375189695908689730230717428916617325031998926780420806061386666046378583550071837247219466671697801843275332629753429308619047895535595684965817408085518888469910885027747090610603268324756078382749082990796132294651696881962928843629666725462682052451754753210899245133102912862225695572223573065210237545360047657039153983266072141586295117635668640223145638447889078773869374500518526813523569610350738113091270685978352989525552843022947752887419320518566118681697782148121899491792256309190853432950405647536031737543390421102237554138964154770691717834238142835295903088016342454100425770493220937550307457687387407409126952330334735420567375549795107901955493785165495953409323894060830435620570206635849734662458737137483566337649534813334036348785228898195429450462856437144411867635374055751693375071803133807123831194651089909562997282413021342896561111606763292751884999306433844319748614157717565114832200109899469975816243813734116195852267016018802776752909623986709057339404407174549479917623262292240692002948370839150438273618393548860906396549033845355030052330412431992551370306302513601841256455352161676868724062742771044714295698520709295655308658285122710482792611122497388344593857474560674622871831336277084951405000971113340387563469498883887914139569735935236194021553028097485092302193506412589956409691851635642983987557063969527137748846851575268282918423430404714076185770651479593680730853234713962389176638219778886764674683880048309236507035373824040232233029653571112335890752198564060769684218360193945775160795150387853632496549813389083360477029277096925068714425757948063374241049937951042803950342033445371303446297868003510919059361392419647184606546262096418104251841412863845536203737205379212842289342292425511612761797538648763406643439484569320659994291100675343273494669834294803734900203737013417100118038378603369901381043977052704473311759416532482038269394632551113175893258270851602265446447153690360333251382016326750069857718712564112021845023970168381605698323370567169959382223112364162500269668256178549619453546360694675626436892488960735152429417700094916671647123637211445155968709499281655596987120695031551539821456765527088897416280303321025535004008268548802003627233782548010446344265286141915222903410738787642286337933982028810804614620125176438395819871970310118074213659161270060303320147119740836883503451883920252341235864479602200143227174386696981900286498662348078264892352177818224004514752719165951094725795533280934772262927043111481494055926359424235970372517366815655300580013381348371759046046757473872400568034434821922993214630315067626807995383783042059170832893457171236288972797216424484243452006314419875885003142696934614100686515162788694105289772813597152167946168786411249023743671931125014764497751746904121124768439753373523227956202640430097973548435354421886542664469609014595268032114191836055029851062320977153773519695423715145616656763470440175259897425518305743905458086381771427323082710416295272874800283981121797120387330285020531180553330873675973570329780940699892742659404700958312842654586541380065209576923861953494953244980589742400910631653909675098367373824932315128757405422051078783149635657283367454784236047812606123857121673521141126766180764298752559700776860395728257996327358401478326259259581959799308176211406304030132166284562177971066260407527995949612839043009242449428647277062867168836292110234255156931233274729663376277524208114248267584966459255180803428756039291103438018453955386531039465367863091803176115996981580478418519447947178777991641391483748965509969338227280424957069644781368246006940789745400817158831463264620202428306475038428611078677821771576377491124868731029308476997399021578735567393066015642577983921537588625044630586559382290440763847114321212536067418322983439160609583436016154595162039972480980008332228395199864804623366206046287202278433837789586772432779453559606141046902293501407679939567136846571571044081627160686123718818620365067743226481257586704522628234395828674169980644108757404872121179081250052105606280632371512636304875614875331673091940145417773319286941057345915626255651256729555241877906619662397348792553925012003504573694580532849897401247362118653720468736926707714573232436483875436639567948796062100481620396560271087641548481214783088169791478644511421343721492599932808338944442497931996750503707235112639922385857006307374476783681421229116265172974206771525132488764702906253909587073474212412935113386687248996231488463264927114176176606782589023428441986080284180720406177115214452516286888557188451460815210496326837237342356812415596479229526963796184718103077582460237516373032222533111717015708360009181166186690893761125871866251795808680194454675535629991894668957901068786133880222894562528879618149779133631559413294408508925660066552175982654999623437731390458892269796380923510697846674851511524236432846015453841617878844151495980039563576277468710559559504734256804555726188729215422981580796997679996581269745921191688088036166307731239199049156530789760311766256833468490055725653931258165633614413547709825554792482801243007903177214026017284086802906815362569255783313807919992331182775130760898847381443827489130336791007435996494988663449256356596407576032037042868049377909937976049387851484409322778626321293144713204042513873194660105633232156901765941969751014202828772356314444902651251269300918644946117200428049344749431282107095654700352461861678601623592735881068759036403473880535373166022301274570514276804616802944429547142667545675143272142233592045135545713589472102792508441269682856084437121442734453596431049608610221712876070764938673298309988081317006986451486585275957574350389443844496652865513850933422222636435646808265687072317060243536366215919702941680133258636467236343604460913909796315104891664793520407151458107200330377393189336723045525364105197618678446866094876776848855145935421068170006587555400643512387023828224954504917720764100636200870509420992454158888516314733399981887759362941973842609558522566136501600005644156223359471562815428537113376652186934562870261429731322862610015089495425001449434545412102225919924752872028502296356828005239492882941451727218054065987947481435028418402731328638226359122882508844898430328384505492681038657793082287020532873923483089778902549365648252634899913502774099888711917849515178703156937257827721341028407183500575227930534413117949974122387752675340845477960971741691484023286722182936686890443041556517488002348174742780584563327505255949041342212396547936708911843336538611672001659958722287011009586434792648803519492611640809174833433699160739691715399816525889574764376903304639463002159954653568953732482568617001962812913542623874561154246160760917505884553729053860769351026228431265217434495664888734090602779849054111646975721102984229708675731050327792231950360663655358601924534192157063431462199691679554980955937647441387119458510455722292254930602000085017298529943018391539707128115102269103636130391850166105780428390896371390288202528948283189290089607075758795759210654736664310179145536021185898118071999824826475858149963386764096920104575668186308592692256147392895871823073456433440211252581338448477237978225232410058902652561385066844893333838747728803852386387116824664646824817016036916255256755661471496476513928593768702338738478831308164254414245655118209131884370225087353169977898716573363356970996025441219355836090236496022132391110897791070642850110249138078715353935306599126752466095648124899980927786228296878603245860532827519988760643395988696926590071732290612978947566092627148724282313454981871542909915965930571941721513364476016868409145994772071192490287994289503839837245950902627121243405749749445336817
p=8

and you'll get a perfect score on the public eval set of ARC-AGI-2! (Feel free to scroll horizontally. Only the first 10,000 digits of ||(\alpha||) are shown.) This number is 260,091 digits long and is effectively god in box, right? One scalar value that cracks one of the most challenging AI benchmarks of our time. Sounds pretty impressive, right? Unfortunately, it's complete nonsense. There is no learning or generalization. What I've really done here is train on the public eval set of ARC-AGI-2 and then use some clever mathematics from chaos theory to encode all the answers into a single, impossibly dense parameter. Rather than a breakthrough in reasoning, it's a very sophisticated form of cheating. The model scores 100% on the public eval set of ARC-AGI-2 but would score 0% on the private eval set of ARC-AGI-2. This one-parameter model is a way to explore some cool math, an absurd thought experiment taken seriously. Yet as we unravel the surprisingly rich mathematics underlying the one-parameter model, it opens up deeper discussions about generalization, overfitting, and how we should actually be measuring machine intelligence in the first place. Let me show you how it works.

ARC-AGI

"Intelligence is measured by the efficiency of skill-acquisition on unknown puzzles. Simply, how quickly can you learn new skills?" - ARC-AGI creators

Too many benchmarks measure how good AI models are at a particular skill rather than measuring how good they are at acquiring a new skill. ARC-AGI-1 tries to address this by measuring how well AI models can generalize to unseen puzzles. More recently, ARC-AGI-2 was released as a more challenging follow up to ARC-AGI-1. ARC-AGI-2 will be the focus of this blogpost. ARC-AGI-2 consists of visual grid-based reasoning puzzles, similar to an IQ-test. Each puzzle provides several example images that demonstrate an underlying rule and a test image that requires applying that rule. Each image is an n x m matrix (list of lists) of integers between ||(0||) and ||(9||) where ||(1 \leq n, m \leq 30||). To display an image, we simply choose a unique color for each integer. As an example,

This puzzle contains 3 example input-output pairs that demonstrate the rule. Given these 3 examples and the question input, we have to infer the hidden rule and predict the question output. Here the hidden rule is straightforward: translate the red shape in a straight line toward the blue shape until they touch (but do not overlap); do not move the blue shape. Indeed, the solution is

Here is another puzzle

The hidden rule is to take the three diagonal lines in the input and repeat them cyclically across all diagonals in the output, forming a repeating checkered pattern. Given the three examples and the question input, the question output is

There are hundreds of puzzles like this in ARC-AGI-2. Solving each puzzle requires deducing new patterns and generalizing to unforeseen puzzles, something it is quite hard for the current crop of AI models.

Performance on private eval set of ARC-AGI-2. Retreived from https://arcprize.org/leaderboard on December 5th, 2025.

Even the world's best models struggle on ARC-AGI-2, all scoring under ||(50\%||). Gemini 3 Deep Think (Preview) has the highest score of ||(45.1\%||) but costs a staggering ||(\$77.16||) per puzzle. GPT-5 Pro is much more efficient, costing ||(\$7.14||) per puzzle but only solving ||(18.3\%||) of puzzles. Many other frontier models -- Claude, Grok, and Deepseek can't even crack ||(20\%||). In contrast, a human panel gets ||(100\%||) of questions right. That's why there exists a ||(\$1,000,000||) competition to open source a solution to ARC-AGI-2. It's that difficult.

The HRM Drama

In July, HRM released a 27M parameter model inspired by the brain's "slow" and "fast" loops. It scored 40.3% on ARC-AGI-1, crushing larger models like o3-mini-high (34.5%) and Claude-3.7-8k (21.2%).

HRM scores on public eval set of ARC-AGI-1 and ARC-AGI-2.

The results almost seemed to be too good to be true. How can a tiny 27M parameter model from a small lab be crushing some of the world's best models, at a fraction of their size? Turns out, HRM trained on the examples, not questions, of the public eval set.

Screenshot of HRM paper showing that HRM trained on the public eval set of ARC-AGI-1.

Does this actually count as "training on test"? The HRM authors never actually trained on the the questions used to measure model performance, just the examples associated with them. This contraversy set AI twitter on fire [1, 2, 3 4 5]!

The ARC-AGI organizers ultimately accepted the HRM submission, indicating it is fine to train on the examples of the public eval set Twitter. But buried in a GitHub thread, HRM's lead author, Guan Wang, made an offhand comment that caught my attention:

"If there were genuine 100% data leakage - then model should have very close to 100% performance (perfect memorization)." - Guan Wang

That line stuck with me. If partial leakage gets you ||(40.3\%||) on ARC-AGI-1, what happens with complete leakage? If we train on the actual eval questions, not just eval examples, can we hit ||(100\%||)? Can we do it with even fewer parameters than HRM (27M) or TRM (7M)? And can we do it on the more challenging ARC-AGI-2 instead of ARC-AGI-1? How far can we push this?

Chaos Theory

"Chaos is what killed the dinosaurs, darling." - J.D. in Heathers

My goal was simple: create the tiniest possible model that scores 100% on ARC-AGI-2 by blatantly training on the public eval set, both the examples and questions. We would deviate from HRM's acceptable approach (training on just the examples of the public eval set) and enter the morally dubious territory of training on the examples and questions of the public eval set. Now, the obvious approach would be to build a dictionary - just map each input directly to its corresponding output. But that's boring and lookup tables aren't nice mathematical functions. They're discrete, discontinuous, and definitely not differentiable. We need something else, something more elegant and interesting. To do that, we are going to take a brief detour into the world of chaos theory.

Note: As far as I know, Steven Piantadosi pioneered this technique in One parameter is always enough. Yet I first heard of it through Laurent Boué's paper Real numbers, data science and chaos: How to fit any dataset with a single parameter. This paper is really a gem due its sheer creativity.

In chaos theory, the dyadic map ||(\mathcal{D}||) is a simple one-dimensional chaotic system defined as

||[ \begin{align} \mathcal{D}: [0, 1] \to [0, 1] && \mathcal{D}(a) &= (2a) \bmod 1. \tag{2} \end{align} ||]

It takes in any number between 0 and 1, doubles it, and throws away the whole number part, leaving just the fraction. That's it.

In chaos theory, we often study the orbit or trajectory of a chaotic system, the sequence generated by applying the chaotic map to itself over and over again. Starting with some number ||(a||), we apply our map to get ||(\mathcal{D}(a)||), and again to get ||(\mathcal{D}(\mathcal{D}(a))||), and so on and so forth. Let

||[ \begin{align*} \mathcal{D}^k(a) & := \underbrace{(D \circ ... \circ D)}_{k}(a) = (2^k a) \mod 1 \tag{3} \end{align*} ||]

mean we apply the dyadic map ||(k||) times to ||(a||). What does the orbit ||((a, \mathcal{D}^1(a), \mathcal{D}^2(a), \mathcal{D}^3(a), \mathcal{D}^4(a), \mathcal{D}^5(a))||) look like?

If ||(a = 0.5||), the orbit is ||((0.5, 0.0, 0.0, 0.0, 0.0, 0.0)||).
If ||(a = 1/3||), the orbit is ||((0.333, 0.667, 0.333, 0.667, 0.333, 0.667,)||)
If ||(a = 0.431||), the orbit is ||((0.431, 0.862, 0.724, 0.448, 0.897, 0.792)||)

One orbit seems to end in all zeros, another bounces back and forth between ||(0.333||) and ||(0.667||), and a third seems to have no pattern at all. On the surface, these orbits do not have much in common. But if we take a closer look, they all share the same underlying pattern. Let's revisit the third orbit for ||(a = 0.431||) but this time we will analyze its binary representation:

Iterations	Decimal	Binary	Observation
0	\|\|(a = 0.431\|\|)	\|\|(\text{bin}(a) = 0.011011...\|\|)	Original number
1	\|\|(D^1(a) = 0.862\|\|)	\|\|(\text{bin}(D^1(a)) = 0.11011...\|\|)	First bit of \|\|(a\|\|) \|\|((0)\|\|) removed
2	\|\|(D^2(a) = 0.724\|\|)	\|\|(\text{bin}(D^2(a)) = 0.1011...\|\|)	First two bits of \|\|(a\|\|) \|\|((01)\|\|) removed
3	\|\|(D^3(a) = 0.448\|\|)	\|\|(\text{bin}(D^3(a)) = 0.011...\|\|)	First three bits of \|\|(a\|\|) \|\|((011)\|\|) removed
4	\|\|(D^4(a) = 0.897\|\|)	\|\|(\text{bin}(D^4(a)) = 0.11...\|\|)	First four bits of \|\|(a\|\|) \|\|((0110)\|\|) removed
5	\|\|(D^5(a) = 0.792\|\|)	\|\|(\text{bin}(D^5(a)) = 0.1...\|\|)	First four bits of \|\|(a\|\|) \|\|((01101)\|\|) removed

Looking at the Binary column, we see that every time we apply the dyadic map, the most significant bit is removed! We start off with ||(0.011011||), and then applying ||(\mathcal{D}||) once removes the leftmost ||(0||) to get ||(0.11011||), and applying ||(\mathcal{D}||) another time removes the leftmost ||(1||) to get ||(0.1011||). Although the orbit appears irregular in its decimal representation, a clear pattern emerges from the binary representation. What is going on here? Each time we call ||(D(a) = (2a) \mod 1||), we double and truncate ||(a||). The doubling shifts every binary digit one place to the left and the truncation throws away whatever digit lands in the one's place. In other words, each application of ||(\mathcal{D}||) peels off the first binary digit and throws it away. If we apply the dyadic map ||(k||) times, we remove the first ||(k||) bits of ||(a||). We can see this process also holds for our other orbits:

If ||(a = 0.5||), we get the orbit ||((0.5, 0.0, 0.0, 0.0, 0.0, 0.0)||) because ||(\text{bin}(a) = 0.100000...||) and after discarding the first bit, which is a ||(1||), we are left with all zeros.
If ||(a = 1/3||), we get the orbit ||((0.333, 0.667, 0.333, 0.667, 0.333, 0.667)||) because ||(\text{bin}(a) = 0.010101...||), an infinite sequence of bits alternating between ||(1||) and ||(0||). When the bits start with a 0, we get ||(0.010101...||) which is ||(1/3 = 0.333||) in decimal. And when the bits start with a ||(1||), ||(0.10101...||), we get ||(2/3 = 0.667||) in decimal.

Remarkably, these orbits are all governed by the same rule: remove one bit of information every time the dyadic map is applied. As each application of ||(\mathcal{D}||) removes another bit, this moves us deeper into the less significant digits of our original number -- the digits that are most sensitive to noise and measurement errors. A tiny change in ||(a||) due to noise, affecting the least significant bits of ||(a||), would eventually bubble up to the surface and completely change the orbit. That's why this system is so chaotic -- it is incredibly sensitive to even the smallest changes in the initial value ||(a||). (Note: we always compute the dyadic map on decimal numbers, not binary numbers; however, conceptually it is helpful to think about the binary representations of the orbit.)

The Dyadic Map As An ML Model

"When I grow up, I'm going to be a real ~~boy~~ ML Model" - the Dyadic Map if it were staring in Pinacoi

We've discovered something remarkable: each application of ||(\mathcal{D}||) peels away exactly one bit. But here's the question: if the dyadic map can systematically extract a number's bits, is it possible to put information in those bits in the first place? What if we encode our dataset into a number's bits (model.fit) and then use the dyadic map as the core of a predictive model, extracting out the answer bit by bit (model.predict)? In other words, can we turn the dyadic map into an ML model?

A Worked Example

Suppose our dataset contains the three numbers we saw before

||[ \mathcal{X} = \{x_0, x_1, x_2\} = \{0.5, 1/3, 0.431\}. ||]

Let's convert each number to binary and look at the first ||(p=6||) binary digits for simplicity:

||[ \mathcal{B} = \{b_0, b_1, b_2\} = \{ \text{bin}_6(x_0), \text{bin}_6(x_1), \text{bin}_6(x_2)\} = \{0.100000, 0.010101, 0.011011\} ||]

where the function ||(b_i = \text{bin}_p(x_i)||) converts decimal numbers to ||(p||)-bit binary numbers. Now comes the clever part: we glue these binary strings together, end to end:

||[ b = 0. \underbrace{100000}_{b_0} \underbrace{010101}_{b_1} \underbrace{011011}_{b_2} ||]

and convert this binary string back to decimal

||[ \alpha = \text{dec}(b) = 0.50522994995117188 ||]

The number ||(\alpha||) is carefully engineered so that it is a decimal number whose bits contain our entire dataset's binary representation. That's right: we've just compressed our entire dataset into a single decimal number! We only have one parameter, not billions here! This is a very simple, stupid version of ||(\alpha = \text{model.fit}(\mathcal{X})||). But here's the question: given ||(\alpha||), how do we get our data ||(\mathcal{X}||) back out? How do we do ||(\tilde{x}_i = \text{model.predict}(\alpha)||)? This is where the dyadic map becomes our extraction tool. Step 1. Trivially, we know the first 6 bits of ||(\alpha||) contains ||(b_0||).

||[ \begin{align*} \alpha &= 0.50522994995117188 \\ \text{bin}(\alpha) &= 0.\underbrace{100000}_{b_0}\underbrace{010101}_{b_1}\underbrace{011011}_{b_2} = 0.100000010101011011 \end{align*} ||]

So we'll just record the first ||(6||) bits of ||(\alpha||) to get ||(b_0||).

||[ \begin{align*} b_0 &= \text{bin}(\alpha)_{0:6} = 100000 \end{align*} ||]

If we convert this number ||(b_0||) back to decimal, we'll recover our original data, up to the first ||(6||) digits of precision.

||[ \begin{align*} \tilde{x}_0 &= \text{dec} ( b_0 ) = 0.500000 \end{align*} ||]

Now from ||(\alpha||) we've extracted the prediction ||(\tilde{x}_0 = 0.500000||) which matches exactly the ||(0||)th sample of our dataset ||(x_0 = 0.5||). Step 2. To predict the next number, ||(\tilde{x}_1||), remember that each application of ||(\mathcal{D}||) strips away the leftmost binary digit. So

||[ \begin{align*} D^6(\alpha) &= 0.334716796875 \end{align*} ||]

strips away the first ||(6||) bits of ||(\alpha||), which just removes ||(b_0||), and leaves us with ||(b_1, b_2||)

||[ \begin{align*} \text{bin}(D^6(\alpha)) &= 0.\underbrace{\hspace{1cm}}_{b_0}\underbrace{010101}_{b_1}\underbrace{011011}_{b_2} = 0.010101011011 \end{align*} ||]

Like before, we'll then record the first ||(6||) bits of ||(D^6(\alpha)||) to get ||(b_1||)

||[ \begin{align*} b_1 &= \text{bin}(D^6(\alpha))_{0:6} = 010101 \end{align*} ||]

and convert ||(b_1||) back to decimal to get ||(\tilde{x}_1||)

||[ \begin{align*} \tilde{x}_1 &= \text{dec} (b_1) = 0.328125 \end{align*} ||]

Here our prediction ||(\tilde{x}_1 = 0.328125||) is slightly off from the true value ||(x_1 = 1/3||) due to the limits of ||(6||)-bit precision. If we'd have more digits of precision and increase ||(p||), ||(\tilde{x}_1||) would be closer to ||(x_1||). Step 3. To get the next number, ||(b_2||), apply ||(\mathcal{D}||) another 6 times to remove a total of ||(12||) bits from ||(\alpha||),

||[ \begin{align*} D^{12}(\alpha) &= 0.421875 \end{align*} ||]

which strips off ||(b_0, b_1||) and leaves us with just ||(b_2||)

||[ \begin{align*} \text{bin}(D^{12}(\alpha)) &= 0.\underbrace{\hspace{1cm}}_{b_0}\underbrace{\hspace{1cm}}_{b_1}\underbrace{011011}_{b_2} = 0.011011 \end{align*} ||]

Like before, we'll then record the first ||(6||) bits of ||(D^{12}(\alpha)||) to get ||(b_2||)

||[ \begin{align*} b_2 &= \text{bin}(D^{12}(\alpha))_{0:6} = 011011 \end{align*} ||]

and convert ||(b_2||) back to decimal to get ||(\tilde{x}_2||)

||[ \begin{align*} \tilde{x}_2 &= \text{dec} (b_2) = 0.421875 \end{align*} ||]

Notice again that our prediction ||(\tilde{x}_2 = 0.421875||) is slightly off from the true value ||(x_2 = 0.431||) due to the limitations of ||(6||)-bit precision. Let

||[ \begin{align*} \tilde{\mathcal{X}} &= \big \{\tilde{x}_0, \tilde{x}_1, \tilde{x}_2 \big\} = \big \{ 0.500000, 0.328125, 0.421875 \big \} \end{align*} ||]

be the predictions made by our strange dyadic model. If everything is correct, our predicted dataset ||(\tilde{\mathcal{X}}||) should perfectly equal our original dataset ||(\mathcal{X}||) up to the first ||(p||) bits. These 3 steps are summerized in the table below.

Iteration \|\|(i\|\|)	\|\|(ip\|\|) bits removed	\|\|(\mathcal{D}^{ip}(\alpha)\|\|) in decimal	\|\|(\mathcal{D}^{ip}(\alpha)\|\|) in binary	\|\|(b_i\|\|), the first \|\|(p\|\|) bits of \|\|(\mathcal{D}^{ip}(\alpha)\|\|) in binary	\|\|(\tilde{x}_i\|\|), the first \|\|(p\|\|) bits of \|\|(\mathcal{D}^{ip}(\alpha)\|\|) in decimal
\|\|(0\|\|)	\|\|(0 \cdot 6 = 0\|\|)	\|\|(\alpha = 0.50522994995117188\|\|)	\|\|(\text{bin}(\alpha) = 0.\underbrace{100000}_{b_0}\underbrace{010101}_{b_1}\underbrace{011011}_{b_2}\|\|)	\|\|(b_0 = 010101\|\|)	\|\|(\tilde{x}_0 = 0.500000\|\|)
\|\|(1\|\|)	\|\|(1 \cdot 6 = 6\|\|)	\|\|(\mathcal{D}^6(\alpha) = 0.33471679687500000\|\|)	\|\|(\text{bin}(D^6(\alpha)) = 0.\underbrace{\hspace{1cm}}_{b_0}\underbrace{010101}_{b_1}\underbrace{011011}_{b_2}\|\|)	\|\|(b_1 = 010101\|\|)	\|\|(\tilde{x}_1 = 0.328125\|\|)
\|\|(2\|\|)	\|\|(2 \cdot 6 = 12\|\|)	\|\|(\mathcal{D}^{12}(\alpha) = 0.42187500000000000\|\|)	\|\|(\text{bin}(D^{12}(\alpha)) = 0.\underbrace{\hspace{1cm}}_{b0}\underbrace{\hspace{1cm}}_{b1}\underbrace{011011}_{b_2}\|\|)	\|\|(b_2 = 011011\|\|)	\|\|(\tilde{x}_2 = 0.421875\|\|)

In decimal, we go from ||(\alpha = 0.50522994995117188||) to ||(\mathcal{D}^6(\alpha) = 0.33471679687500000||) and then to ||(\mathcal{D}^{12}(\alpha) = 0.42187500000000000||). Although this pattern looks completely random, we are shifitng bits with superb precision. This is anything but random. Think about what we've accomplished here. We just showed that you can take a dataset compress it down to a single real number, ||(\alpha||). Then, using nothing more than repeated doubling and truncation via ||(\mathcal{D}||), we can perfectly recover every data point ||(\tilde{\mathcal{X}}||) up to ||(p||) bits of precision. The chaotic dynamics of the dyadic map, which seemed like a nuisance, turns out to be the precise mechanism we need to systematically access the desired information.

The Algorithm

The algorithm itself is deceptively simple once you see the pattern:

Encoding Algorithm ||(g(p, \mathcal{X})||): Given a dataset ||(\mathcal{X} = \{x_0, ..., x_{n-1}\}||) where ||(x_i \in [0, 1]||) and precision ||(p||), encode the dataset into ||(\alpha||):

Convert each number to binary with ||(p||) bits of precision ||(b_i = \text{bin}_p(x_i)||) for ||(i=0, ..., n-1||)
Concatenate into a single binary string ||(b = b_0 \oplus ... \oplus b_{n-1}||)
Convert to decimal ||(\alpha = \text{dec}(b)||)

The result is a single, decimal, scalar number ||(\alpha||) with ||(np||) bits of precision that contains our entire dataset. We can now discard ||(\mathcal{X}||) entirely.

Decoding Algorithm ||(f_{\alpha, p}(i)||): Given sample index ||(i \in \{0, ..., n-1\}||), precision ||(p||), and the encoded number ||(\alpha||), recover sample ||(\tilde{x_i}||):

Apply the dyadic map ||(\mathcal{D}||) exactly ||(ip||) times ||(\tilde{x}'_i = \mathcal{D}^{ip}(\alpha) = (2^{ip} \alpha) \mod 1||)
Extract the first ||(p||) bits of ||(\tilde{x}'_i||)'s binary representation ||(b_i = \text{bin}_p(\tilde{x}'_i)||)
Covert to decimal ||(\tilde{x}_i = \text{dec}(b_i)||)

Mathematically, we can express these two algorithms with an encoder function ||(g: [0, 1]^n \to [0, 1]||) that compresses the dataset and a decoder function ||(f: \overbrace{[0, 1]}^{\alpha} \times \overbrace{\mathbb{Z}_+}^{p} \times \overbrace{[n]}^i \to [0, 1]||) that extracts individual data points:

||[ \begin{align*} \alpha &= g(p, \mathcal{X}) := \text{dec} \Big( \bigoplus_{x_i \in \mathcal{X}} \text{bin}_p(x_i) \Big) \tag{4} \\ \tilde{x}_i &= f_{\alpha, p}(i) := \text{dec} \Big( \text{bin}_p \Big( \mathcal{D}^{ip}(\alpha) \Big) \Big) \end{align*} ||]

where ||(\oplus||) means concatenation. The precision parameter ||(p||) controls the trade-off between accuracy and storage efficiency. The larger ||(p||) is, the more accurately our encoding, but the more storage it takes up. Our error bound is

||[ |\tilde{x}_i - x_i | < \frac{1}{2^p} ||]

because we don't encode anything after the first ||(p||) bits of precision. What makes this profound is the realization that we're not really "learning" anything in any conventional sense. We're encoding it directly into the bits of a real number, exploiting it's infinite precision, and then using the dyadic map to navigate through that number and extract exactly what we need, when we need it. From this perspective, the dyadic map resembles a classical ML model where the encoder ||(g||) acts as model.fit() and the decoder ||(f||) acts as model.predict().

Applying Some Makeup

"You don’t want to overdo it with too much makeup" - Heidi Klum

How do we go from the ugly, discontinuous decoder function

||[ f_{\alpha,p}(i) := \text{dec} \Big( \text{bin}_p \Big( \mathcal{D}^{ip}(\alpha) \Big) \Big) ||]

to that beautiful function I promised you at the start of the blog

||[ f_{\alpha, p}(i) = \sin^2 \Big( 2^{i p} \arcsin^2(\sqrt{\alpha}) \Big) ? ||]

In this section we will "apply makeup" to the first function to get it looking a bit closer to the second function. We will keep the same core logic but make the function more ascetically pleasing. To do this, we will need another one-dimensional chaotic system, the logistic map.

Logistic Map

The logistic-map at ||(r=4||) on the unit interval is defined as

||[ \begin{align*} \mathcal{L}: [0, 1] \to [0, 1] && \mathcal{L}(a_L) &= 4 a_L (1 - a_L) \tag{6} \end{align*} ||]

which seems quite different than the familiar dyadic map

||[ \begin{align*} \mathcal{D}: [0, 1] \to [0, 1] && \mathcal{D}(a_D) &= (2 a_D) \mod 1 \end{align*} ||]

One is a bit-shifting operation, the other is a smooth parabola that ecologists use to model population growth. (Note: previously ||(a||) was the input to the dyadic map but from now on ||(a_D||) will be the input to the dyadic map to differentiate it from ||(a_L||), the input to the logistic map.)

What does the logistic orbit ||((a_L, \mathcal{L}^1(a_L), \mathcal{L}^2(a_L), \mathcal{L}^3(a_L), \mathcal{L}^4(a_L), \mathcal{L}^5(a_L))||) look like? Similar or different to the dyadic orbit ||((a_D, \mathcal{D}^1(a_D), \mathcal{D}^2(a_D), \mathcal{D}^3(a_D), \mathcal{D}^4(a_D), \mathcal{D}^5(a_D))||)?

If ||(a_L = a_D = 0.5||), the logistic orbit is ||((0.5, 1.0, 0.0, 0.0, 0.0, 0.0)||) while the dyadic orbit is ||((0.5, 0.0, 0.0, 0.0, 0.0, 0.0)||). Although the dyadic orbit is just all zeros, the logistic orbit actually has ||(1.0||) as the second number in the orbit.
If ||(a_L = 1/3||), the logistic orbit is ||((0.333, 0.888, 0.395, 0.956, 0.168, 0.560)||) while the dyadic orbit is ||((0.333, 0.667, 0.333, 0.667, 0.333, 0.667)||). While the dyadic orbit repeats in a simple pattern, the logistic orbit is seemingly patternless.
If ||(a_L = 0.43085467085||), the logistic orbit is ||((0.431, 0.981, 0.075, 0.277, 0.800, 0.639)||) while the dyadic orbit is ||((0.431, 0.862, 0.724, 0.448, 0.897, 0.792)||). Both orbits here seem totally random and chaotic, but each in their own way.

The logistic and dyadic maps create orbits that look nothing alike! However, topological conjugacy tells us these two maps are actually the same. Not similar. Not analgous. The same. They have identical orbits, the exact same chaotic trajectories, simply expressed in different coordinates. The logistic map, for all its smooth curves and elegant form, is actually doing discrete binary operations under the hood, just like the dyadic map (and vice versa). Formally, two functions are topologically conjugate if there exists a homeomorphism, fancy talk for a change of coordinates, that perfectly takes you from one map to another. The change of coordinates here is

||[ \begin{align*} a_L = \phi(a_D) &= \sin^2(2 \pi a_D) \tag{7} \\ a_D = \phi^{-1}(a_L) &= \frac{1}{2 \pi} \arcsin (\sqrt{a_L}) \tag{8} \end{align*} ||]

where ||(\phi: [0, 1] -> [0, 1]||) and ||(\phi^{-1}: [0, 1] -> [0, 1]||). We can map any ||(a_L||) to an ||(a_D||) and any ||(a_D||) to an ||(a_L||).

Looking at the plot, the function ||(\phi||) has a period of 1, meaning it repeats the same values every time ||(a_D||) increases by ||(1||). This periodicity is crucial because it allows us to drop the modulo operation from the dyadic map ||(\mathcal{D}(a_D) = (2 a_D) \mod 1||) when transforming from the dyadic space to the logistic space. Formally,

||[ \begin{align*} \phi(a_D \mod 1) = \phi(a_D) \tag{9} \end{align*} ||]

which will be important later on. To go back and forth between the dyadic and logistic maps, we apply ||(\phi||) to the output ||(\mathcal{D}||) and get ||(\mathcal{L}||); we can also apply ||(\phi^{-1}||) to the input ||(a_L||) to get ||(\mathcal{D}||). Mathemtically,

||[ \begin{align*} \mathcal{L}(a_L) &= \phi(\mathcal{D}(a_D)) \\ \mathcal{D}(a_D) &= \mathcal{L}(\phi^{-1}(a_L)) \end{align*} ||]

Here ||(\phi||) takes us to the logistic space and ||(\phi^{-1}||) takes us back to the dyadic space. This is astonishing! ||(\phi||) is just a sin wave squared and with a period of one. It's inverse, ||(\phi^{-1}||) is even weirder looking with an ||(\arcsin||). But somehow these functions allow us to bridge the two maps ||(\mathcal{D}||) and ||(\mathcal{L}||)! Moreover, ||(\phi||) and ||(\phi^{-1}||) perfectly relate every single point in the infinite orbits of ||(\mathcal{D}||) and ||(\mathcal{L}||):

||[ (a_D, \mathcal{D}^1(a_D), \mathcal{D}^2(a_D), ...) = (a_L, \mathcal{L}^1(\phi^{-1}((a_L)), \mathcal{L}^2(\phi^{-1}((a_L)), ...) ||]

or it can be expressed as

||[ (a_L, \mathcal{L}^1(a_L), \mathcal{L}^2(a_L), ...) = (a_D, \phi(\mathcal{D}^1(a_D)), \phi(\mathcal{D}^2(a_D)), ...) ||]

depending on if we want to be natively using the coordinate system of ||(\mathcal{D}||) or ||(\mathcal{L}||). What appears as chaos in one coordinate system manifests as the exact same chaos in the other, no matter how many iterations we apply. Mathematically, this suggests something stronger:

||[ \begin{align*} \mathcal{L}^k(a_L) &= \phi(\mathcal{D}^k(a_D)) \tag{10} \\ \mathcal{D}^k(a_D) &= \mathcal{L}^k(\phi^{-1}(a_L)) \tag{11} \end{align*} ||]

Think of these two orbits existing in parallel universes with ||(\phi||) and ||(\phi^{-1}||) acting as the bridges between ||(\mathcal{D}||) and ||(\mathcal{L}||).

Topological conjugacy between the dyadic and logistic map.

We can now revist the dyadic and logistic orbits when ||(a_D = a_L = 0.431||).

When ||(a_D = a_L = 0.431||), we know the dyadic orbit is ||((0.431, 0.862, 0.724, 0.448, 0.897, 0.792)||). If we apply ||(\phi||) to every output element ||(\phi(\mathcal{D}^k(a_D))||), we get ||((0.431, 0.981, 0.075, 0.277, 0.800, 0.639)||). This is exactly the logistic orbit we saw in eqn. (10)!
When ||(a_D = a_L = 0.431||),, we know the logistic orbit ||((0.431, 0.981, 0.075, 0.277, 0.800, 0.639)||). If we apply ||(\phi^{-1}||) to every input element before the logistic map ||(\mathcal{L}(\phi^{-1}(a_L)))||) we get the dyadic orbit ||((0.431, 0.862, 0.724, 0.448, 0.897, 0.792)||).

We see that although both these orbits look completly unrelated, these two orbits are perfectly connected to one another through ||(\phi||) and ||(\phi^{-1}||).

A New Algorithm

Can we use the topological conjugacy of ||(\mathcal{D}||) and ||(\mathcal{L}||) as makeup? While ||(\mathcal{D}||) is ugly and discontinuous, ||(\mathcal{L}||) is smooth and differentiable. We can use the logistic map as "makeup" to hide the crude dyadic operations. We want our decoder to use ||(\mathcal{L}||) instead of ||(\mathcal{D}||). But for the encoder to glue together the bits of our dataset, we needs to be in the dyadic space so our clever bit manipulations will still work out. Here's the strategy:

Encoder: Work in dyadic space where bit manipulation works (use ||(\phi||)) but output parameter in logistic space (use ||(\phi^{-1}||))
Decoder: Work entirely in smooth logistic space using the conjugacy relationship

This gives us two new beautiful encoder/decoder algorithms where the main changes are bolded:

Encoding Algorithm ||(g(\mathcal{X}, p)||): Given a dataset ||(\mathcal{X} = \{x_0, ..., x_n\}||) where ||(x_i \in [0, 1]||) and precision ||(p||), encode the dataset into ||(a_L||):

Transform data to dyadic coordinates: ||(z_i = \phi^{-1}(x_i) = \frac{1}{2 \pi} \arcsin⁡( x_i )||) for ||(i=1, ..., n||)
Convert each transformed number to binary with ||(p||) bits of precision: ||(b_i = \text{bin}_p(z_i)||) for ||(i=1, ..., n||)
Concatenate into a single binary string ||(b = b_0 \oplus ... \oplus b_n||)
Convert to decimal ||(a_D = \text{dec}(b)||)
Transform to logistic space: ||(\alpha = a_L = \phi(a_D) = \sin^2(2 \pi a_D)||)

The result is a single, decimal, scalar number ||(\alpha||) with ||(np||) bits of precision that contains our entire dataset. We can now discard ||(\mathcal{X}||) entirely.

Decoding Algorithm ||(f_{\alpha, p}(i)||): Given sample index ||(i \in \{0, ..., n-1\}||), precision ||(p||), and the encoded number ||(\alpha||), recover sample ||(\tilde{x_i}||):

Apply the logistic map ||(\mathcal{L}||) exactly ||(ip||) times ||(\tilde{x}'_i = \mathcal{L}^{ip}(\alpha) = \sin^2 \Big(2^{i p} \arcsin^2(\sqrt{\alpha}) \Big)||)
Extract the first ||(p||) bits of ||(\tilde{x}'_i||)'s binary representation ||(b_i = \text{bin}_p(\tilde{x}'_i)||)
Covert to decimal ||(\tilde{x}_i = \text{dec}(b_i)||)

Mathematically, we can express this with a new and improved encoder ||(g||) and decoder ||(f||):

||[ \begin{align*} \alpha &= g(p, \mathcal{x}) := \phi \bigg( \text{dec} \Big( \bigoplus_{x \in \mathcal{X}} \text{bin}_p(\phi^{-1}(x)) \Big) \bigg) \\ \tilde{x}_i &= f_{\alpha,p}(i) := \text{dec} \Big( \text{bin}_p \Big( \mathcal{L}^{ip}(\alpha) \Big) \Big) = \text{dec} \Big( \text{bin}_p \Big( \sin^2 \Big(2^{ip} \arcsin(\sqrt{\alpha}) \Big) \Big) \Big) \end{align*} ||]

where ||(\oplus||) means concatenation. The decoder here is tantalizingly close to the function I promised at the start:

||[ f_{\alpha, p}(i) = \sin^2 \Big( 2^{x p} \arcsin^2(\sqrt{\alpha}) \Big) ||]

but is still wrapped with those pesky ||(\text{dec}||) and ||(\text{bin}_p||) operations. However, something profound has happened here. We've taken the crude, discontinuous dyadic map and transformed it into something smooth and differentiable. The logistic map doesn't look like it's doing binary operations, but underneath the elegant trigonometry, it's performing exactly the same bit manipulations as its topological coungant, the dyadic map. Indeed, the makeup looks pretty great! However, nothing is free. The cost of using the logistic map instead of the dyadic map is that our error is now ||(2 \pi||) times larger,

||[ |\tilde{x}_i - x_i | \leq \frac{2 \pi}{2^{p}} = \frac{\pi}{2^{p-1}} ||]

We get this ||(2 \pi||) factor by noting that the derivative of ||(\phi||) is bounded by ||(2 \pi||) and applying the mean-value theorem. For a proof, see section 2.5 of "Real numbers, data science and chaos: How to fit any dataset with a single parameter".

How did we get ||(\mathcal{L}^{ip}(\alpha) = \sin^2 \Big(2^{i p} \arcsin^2(\sqrt{\alpha}) \Big)||)?

We just need to perform some simple algebraic manipulation with our equations:

||[ \begin{align*} \mathcal{L}^k(\alpha) &= \mathcal{L}^k(a_L) & \text{by $\alpha = a_L$} \\ &= \phi(\mathcal{D}^k(a_D)) & \text{by $(10)$} \\ &= \phi((2^k a_D) \mod 1) & \text{by $(3)$} \\ &= \phi(2^k a_D) & \text{by $(9)$} \\ &= \sin^2(2 \pi \cdot (2^k a_D)) & \text{by $(7)$} \\ &= \sin^2 \bigg(2 \pi 2^k \Big( \frac{1}{2 \pi} \arcsin(\sqrt{a_L}) \Big) \bigg) & \text{by $(8)$} \\ &= \sin^2 \Big(2^k \arcsin(\sqrt{a_L}) \Big) & \text{by simplification} \\ &= \sin^2 \Big(2^k \arcsin(\sqrt{\alpha}) \Big) & \text{by $\alpha = a_L$} \end{align*} ||]

Code Implementation

Now comes the moment of truth. We've built up all this beautiful math about chaos theory and topological conjugacy, but can we actually code it up? If you've been paying attention, there is one crucial implementation detail we have to worry about. If our dataset ||(\mathcal{X}||) has ||(n||) samples, each encoded with ||(p||) bits, ||(\alpha||) will contain ||(np||) bits. For ARC-AGI-2 with hundreds of puzzles and high precision, this could be millions of bits. Standard computers can only handle numbers with 32 or 64 bits. How do we even store ||(\alpha||), much less solve ARC-AGI-2 with it? The answer is simple: we can use an arbitrary precision arithmetic library like mpmath that can represent numbers with as many bits as we want. Instead of a regular Python float, we represent ||(\alpha||) as a mpmath float with ||(np||) bits of precision. We then run the decoder with mpmath operations and convert the final result back to a regular Python float. Note: operations with arbitrary precision arithmetic libraries like mpmath tend to be significantly slower than regular floating point operations. But mpmath gives us another gift: it actually removes the pesky ||(\text{dec}(\text{bin}_p(\cdot))||) operations from our decoder

||[ f_{\alpha, p}(i) = \text{dec} \Big( \text{bin}_p \Big( \mathcal{L}^{ip}(\alpha) \Big) \Big). ||]

In our implementation, we use ||(\text{dec}(\text{bin}_p(\cdot))||) to truncate ||(\mathcal{L}^{ip}(\alpha)||) to exactly ||(p||) bits and then we convert ||(f_{\alpha, p}(i)||) from a ||(p||)-bit mpmath number to a Python float32. During this conversion, Python copies the first ||(p||) bits of ||(f_{\alpha, p}(i)||) and then fills the remaining bits of the Python float32 (bits ||(p+1||) through ||(32||)) with random meaningless junk bits (assuming ||(p<=32||)). Since our model only guarantees accuracy for the first ||(p||) bits, these random bits don't matter. However, converting to binary and back is wildly expensive, especially when ||(\alpha||) contains millions of bits. Upon taking a closer look, we can, in fact, actually skip the entire ||(\text{dec}(\text{bin}_p(\cdot))||) step and convert ||(\mathcal{L}^{ip}(\alpha)||) directly to a Python float32. The first ||(p||) bits of ||(\mathcal{L}^{ip}(\alpha)||) still get copied correctly and bits ||(p+1||) through ||(32||) get filled with the higher-order bits of ||(\mathcal{L}^{ip}(\alpha)||) instead of random Python bits. Since our prediction only uses the first ||(p||) bits, these extra bits are irrelevant whether they come from Python junk or from the higher-order bits of our decoder. Removing ||(\text{dec}(\text{bin}_p(\cdot))||), our decoder simplifies to exactly what we promised at the start:

||[ f_{\alpha, p}(i) = \mathcal{L}^{ip}(\alpha) = \sin^2 \Big( 2^{x p} \arcsin^2(\sqrt{\alpha}) \Big) ||]

This is amazing! Usually translating math into code turns beautiful theory into ugly, complicated messes. But surprisingly, leveraging mpmath has the opposite effect and actually makes our decoder even simpler. Now let's get to the code!

Building Blocks

First, we need to import our arbitrary-precision math library, mpmath.

We need some functions to convert from binary to decimal and back. We cannot simply use python's bin function because it only converts integers to binary and we have floats in ||([0, 1]||).

Next we need ||(\phi||) and ||(\phi^{-1}||) to go back and forth between the dyadic and logistic spaces.

We can now implement the logistic encoder

||[ \alpha = g(p, \mathcal{x}) = \phi \bigg( \text{dec} \Big( \bigoplus_{x \in \mathcal{X}} \text{bin}_p(\phi^{-1}(x)) \Big) \bigg) ||]

in code

We compute ||(\alpha||) in five steps using mpmath with precision set to ||(np||) bits. Crucially, step 4 produces an mpmath float with the full ||(np||) bits of precision, which we then transform in step 5 to get our final ||(np||)-bit parameter ||(\alpha||). Next, we implement the logistic decoder

||[ \tilde{x}_i = f_{\alpha, p}(i) = \sin^2 \Big( 2^{x p} \arcsin^2(\sqrt{\alpha}) \Big) ||]

Again, we set the mpmath precision to ||(np||) bits and implement the decoder in a single line using mpmath's arbitrary-precision functions: Sin, Arcsin, and Sqrt. That's it. Our entire encoder and decoder, the heart of our one-parameter model, is just a handful of lines and a bit of beautiful mathematics.

Basic Implementation

To actually run logistic_encoder and logistic_decoder on ARC-AGI-2, we need three adjustments:

Adjustment 1: Supervised learning ARC-AGI-2 is a supervised problem with input-output pairs ||((X,Y)||), but our encoder only handles unsupervised data ||((X)||). Solution: ignore input ||(X||) and only encode the outputs ||(Y||) since those are what we need to memorize.
Adjustment 2: Shape handling Our encoder expects scalars, not matrices. Solution: flatten matrices to lists for encoding and reshape back for decoding. For an m x n puzzle, we decode mn individual elements, running the decoder mn times per puzzle, not once.
Adjustment 3: Data scaling ARC-AGI-2 uses integers ||(0-9||), but our encoder needs values in ||([0,1]||). Solution: use a MinMaxScaler to squeeze the data into the right range during encoding and unscale them during decoding.

Now we can create a one-parameter model for the first ARC-AGI-2 puzzle in the public eval set. ||(X||) contains the 4 examples and the question input

and ||(Y||) is the question output

Let's run logistic_encoder to encode ||(Y||) into ||(\alpha||) using precision ||(p=7||)

Alpha contains ||(1897||) digits (||(6300||) bits). Feel free to scroll horizontally:

alpha=0.74285114332024342325549937083636474565519009327862941959665140406783483205479218836647152731554946432856788703992188906990891731482182847524308776802281798217124682447263086292706558015080117587125137946881353134901846130213659216073847350085266801546500388056303279812229387014566738064635728388315625125612460908053813612115944371403488022330138204005465833529402443403780893627567256369296303433860649253101774272722381473011643003744092414141792080856154336852232751681761870825262961239388416686534036505209391584808776416887122876960233618275222404879749262022119613913874625326198219211198620416420976413170105387014954985331241357748594103136211408700904101070819345678337562478638731082420742643951934007189701107794645826238176372535654425768205812193697418665115645591019081599730450414681394654648740614534913490603387271476765995708667386637327422984549190564023567503974083813261600742256850890763198442464530739947767438158346711984686537123296971853036329532017930470957886617222952038244846792482431147430762987112113198332310791887702977246168036347134814764098024874588331258467793920192910184049083305781018189848769815863752005856715581710276074190641816852407672817742796240720806033022892180099812775701709359799032789650317971424349107886413845671496713331998226307819777294734024746164655829893066785266198516930143907068214149756610237486409825479704403950049986719886018007204851422317073595042856901520808205036921515904367496432996379033889021520978522808476522459195470488476590221986500425992848671776649140155585867456664879520985321530329590064710028585228533299150027623406426839690690773509222974472641729905841013913165150369477959654018728154622535075452933489985516349842376600009501513262596103419929725321662046257221998489659517783086965851309721070204164329693715232879338957676296906129816570061872627789636932222196808375062090735051961248623508890396

This is our one-parameter model in its full glory! This scalar ||(\alpha||) is all we need is to correctly predict the question output of this puzzle! Let's run logistic_decoder to recover ||(Y||) from ||(\alpha||).

Here are the results:

The left column shows the correct answers, and the right column shows what our model predicted. We use different colors to show how big each mistake is: bigger mistakes have bigger color differences. Look at the first row. The left element should be 7 and we predicted 6.69. The middle element should be 7, and we predicted 6.72. The right element should be 9, and we predicted 8.98. Throughout the whole grid, our predictions are close to the correct values, usually off by just a small decimal. What went wrong? These small errors happen because of our precision setting ||(p||). Remember, the encoder saves each number using only ||(p||) bits and throws away everything else. This cuttoff creates quantization errors up to ||(\frac{\pi R}{2^{p-1}} = 0.44||) (||(p=7||) is the precision and ||(R=9||) is the range of the MinMaxScaler). All our errors are indeed less than ||(0.44||). There's nothing broken—this is just what happens when you use limited precision. But we can make the errors smaller by using higher precision. Or make the error larger by using lower precision. Let's train a one-parameter model with ||(p=4||) and another one ||(p=14||). This gives us

P=5
alpha=0.720513558308703618640328591393307557764330936574382792747147164158013805385225876792947742458370890841526727106017186982377802193282496663616112790111484227447638627001821928407067681543901460763496878967882008659769520635557977409943763637913971887447476420204995149681787013296318236694714270696155942477229356166929431428352529386546481515263645920468442049504058850087791435826086914647316280441439117796649455284319996590409037447182365445065972145661011009793928601405408365681514990921247729794068024484607726728378600345294712802918428569030077847933266582638722578592842387536646679714745500714386972544553931816971567786704088816631709221332680402958680110996596288110423292135736715444260228217343348588516391228311471857564527271164799523998255231188909196037997115928054558604569471290293679833763591599293700326525220498537454890648353154414628856236436303301648361981146822987637373601745700037228621386998704811125980029123867544833360797156613190419157292916232960671344633088092512052341683987077366760610376657799304640784082313824799756122347713090108773378904502785140727021123474069209221831258884998951565308624996991324982743983920069760799063858081445584446341782537653193219895071820715112235749200595195165798891056048900786591514714860489953926474850128625079100741462465709180011767126386764464954165896796337707910680211314

P=14
alpha=0.777520986140695809877290611432054488497150630574765379930570415022568399703706683369316902945787001753863091938022769193671379970566844118427937990377064850291631901787584836841419971587497742018741421325594475338293987019129504815695639443472923563020259432448337861279819030482271675012303835858403657617363473612585745448730299890486669654463395533302203107939368687832538283199675875802680074815851984712397399468994292001429594640840485398843989050432493151821835771152091661093408634637217610246816850058051614811727699434149834333393916627330113336357280669853941378295056506328309017855439112797273689272014683786254487940941897529065806842381981026718600951370605303143519597392897061739839122296974894178881292550523751395518887880139949347652493731720587928974366940163226687791110382040729476761995611819772641793712028605646356034292705965924322556494148610409062312182915933436134884317276374303246351587159690173053886753029158101431298183405638648538352284304106836323900459269613944409467469226483965780671792367996413902602712684325045919358068037853273481106629129281287175779995730470595982381533798132789263595588403090395233688739833729707158361614843480499237033955225185239056465334517063842902787949489364016218555574674033570130036454936689560307150933561392102226160375719251021623478829747976074537973892641397910534370386575297529008844382245765423340932083547272711954707182734116459892341464380806164263136340986707806468044377814873965250024835175331312715129689126672760183299314894066607686026131222005344963338192699403605759492653612749591932164262045916064301577638940271324050851664666268087246118010688210991603557518806277268005292776112321801074592963355964253782258926704889303816612767982830810741234715149022808145470847526320743130725592966011622921539171871321929627512496594094226496957060465342515811299576619008674261742833189477956173993224808357132795467562666108141626850213813567052603611102890991290574451373940181586520478907462233796778054303055499734990414917485286631469905577511848430756161771547651022365987900626217827714793631956168541679555034368480629201604543082506129540762963079285937860747270386560163783792136622216576871267119258299305009599290216283740980912224923995601881534922492370845999976588931881760377264716060676351588113910746517647134896641535121289771076049041725691208307998190163776252424371444688932482880398912038383050905916249583123481478314916098176383173207283303889432497154571071000819080421564438390957881910255161947447842865732995438336091691031361435281364594415983359128154001456096489767948129586453105894331404044960906021895111351284009436527038209954522722391514558661498875080275021474257628319192534947144804243410284694196254029793654102682054071895833929214329216393882311618899523920247200563548976420139668963872657666183887092604156598689098855552091322791520494475004132629084793125442408007444066564848600849605668520278776224497414993050697741653706245136897863040863934727251408382256917640865011101819560179977904288048479898530966995558763683522664204778030765232903642722215122057373426970678115485647933051397710840893982342576119798049200920332303835744155907173772837505929983430099040398078508731730967823721141416269540477400720674716560315284430783209146528211785320740934852391116882199253932243053338236787475388242154245254654335948661942082379384902733959018992726634279243916094810421382221873921696729225436191784763060182975587683023886279125973413878417222255062254342611938738373774010402075214209128153493208236202948104765530872460630802090561108299874661073903572250554930585948909415257399347317444860171320505378013859434564881580432780055473598464340955156498696258388476931097883758278663450942610716540004469186278588909632845437161493476119994478934753

We can visually compare these predictions:

With ||(p=14||) every prediction is exactly right (to two decimal places). But there's a tradeoff: we need more storage. The number ||(\alpha||) grows from 1,897 digits to 3,794 digits, or in bits, from ||(900 \times 7=6,300||) to ||(900 \times 14 = 12,600||). The higher we set ||(p||), the more accurate our encoding becomes, but the more storage space it requires. On the flip side, with ||(p=5||), we only need 1,355 digits. But our predictions get much worse. For example, when the correct value is ||(1||), we predict ||(0.43||) or ||(0.34||), which is a closer to ||(0||) than it is to ||(1||). This is a cool tradeoff and illustrates that the one-parameter model actually works!

Faster Implementation

Each question output is a 30×30 grid flattened into a list of 900 scalar numbers (adjustment #2). Since each number is encoded with ||(p||) bits, one question output requires ||(900p||) bits. Encoding all ||(n=400||) question outputs of ARC-AGI-2 into one ||(\alpha||) requires ||(900 \times n \times p = 900 \times 400 \times p=360,000p||) bits. Depending on ||(p||), our ||(\alpha||) can easily reach millions of bits, making it incredibly slow. Moreover, the decoder processes each number separately, so decoding the entire ARC-AGI-2 dataset requires ||(900 \times n = 900 \times 400=360,000||) separate operations. With an ||(\alpha||) that's millions of bits long, this can takes hours. We need to speed this up. Looking at our current decoder

def logistic_decoder(alpha, full_precision, p, i):
    mp.prec = full_precision
    return float(Sin(2 ** (i * p) * Arcsin(Sqrt(alpha))) ** 2)

def decode(alpha, full_precision, p, y_scaled):
    return np.array([logistic_decoder(alpha, full_precision, p, i) for i in tqdm(range(len(y_scaled)), total=len(y_scaled), desc="Decoding")])

y_pred_raw = decode(alpha, full_precision, p, y_scaled)

we can accelerate this in three ways:

Parallelization: Because each number is decoded independently, we can decode all number in parallel with multiprocessing.Pool. This speeds up the for loop over range(len(y_scaled)).
Precomputation: Calculate arcsin(sqrt(alpha)) once before decoding instead of recomputing it every time we call logistic_decoder. This eliminates repeated expensive trigonmetric and square root operations on huge ||(np||)-bit numbers like ||(\alpha||).
Adaptive precision: We currently use all ||(np||) bits of ||(\alpha||) every time we decode as we set mp.prec = full_precision. However, in the ||(i||)th step, we only need the first ||(p(i+1)+1||) bits of ||(\alpha||). Working with smaller numbers drastically reduces the computation needed at each step.

How does adaptive precision work? Why can we use ||(p(i+1)+1||) bits instead of ||(np||) bits in the ||(i||)th decoding step?

Each sample is encoded in ||(p||) bits, so the ||(i||)th sample occupies bits ||(ip||) through ||(ip + (p-1) = p(i+1) - 1||) of ||(\alpha||). The parts of ||(\alpha||) beyond ||(\alpha||) beyond ||(p(i+1) - 1||) bits are irrelevant in iteration ||(i||). By setting mpmath's precision to exaclty ||(p(i+1) - 1||) bits in iteration ||(i||), we perform computation on fewer bits, increasing the precision gradually: ||(p||) bits in iteration ||(0||), ||(2p||) bits in iteration ||(1||), and so on, up to ||(np||) bits in the final iteration. This reduces the total arithmetic cost from ||(n \cdot (np)||) bit-operations to

||[ p(1+2+...+n) = \frac{n(n+1)}{2} p, ||]

which is roughly 2x fewer arithmetic operations. Theoretically this is a constant factor improvement. However, in practice this yields a dramatic speedup in mpmath. A key important caveat is that this optimization only works in dyadic space where the bit structure is explicit. In logistic space, the bit positions are scrambled, making reduced precision unusable. For this reason, we apply reduced precision only after ||(\phi^{-1}||) transforms the value into dyadic space. Shout out to Claude for helping me to debug this nuanced point! Finally, to improve numerical stability, we set mpmath's precision to ||(p(i+1)+1||) bits -- two bits higher than the normal ||(p(i+1)-1||). These two extra bits are not for extracting additional information from ||(\alpha||). Instead, they act as a numerical buffer that helps preserves the accuracy of mpmath’s arithmetic. Emperically, we need this otherwise mpmath does not work properly. I'm not sure why...

Let's implement these three speedups in our code. We define logistic_decoder_fast which takes in arcsin_sqrt_alpha instead of alpha (speedup #2) and sets the precision to mp.prec = p * (i + 1) + 1 instead of mp.prec = full_precision (speedup #3).

We can now define fast_decode which parallelizes the decoding with multiprocessing.Pool (speedup #1).

def fast_decode(alpha, p, y_scaled, n_workers=8):
    y_idxs = list(range(len(y_scaled)))
    decoder = functools.partial(logistic_decoder_fast, Arcsin(Sqrt(alpha)), p)
    with multiprocessing.Pool(n_workers) as p:
        y_pred = np.array(list(tqdm(p.imap(decoder, y_idxs), total=len(y_idxs), desc="Decoding")))
    return y_pred

Let's now run a speed test and compare the fast and slow decoders on 5 ARC-AGI-2 puzzles with precision ||(p=7||). First, we encode these 5 puzzles into ||(\alpha||):

p4=14
len(alpha4)=18966
alpha4=0.7775209861406958098772906114320544884971506305747653799305704150225683997037066833693169029457870017538630919380227691936713799705668441184279379903770648502916319017875848368414199715874977420187414213255944753382939870191295048156956394434729235630202594324483378612798190304822716750123038358584036576173634736125857454487302998904866696544633955333022031079393686878325382831996758758026800748158519847123973994689942920014295946408404853988439890504324931518218357711520916610934086346372176102468168500580516148117276994341498343333939166273301133363572806698539413782950565063283090178554391127972736892720146837862544879409418975290658068423819810267186009513706053031435195973928970617398391222969748941788812925505237513955188878801399493476524937317205879289743669401632266877911103820407294767619956118197726417937120286056463560342927059659243225564941486104090623121829159334361348843172763743032463515871596901730538867530291581014312981834056386485383522843041068363239004592696139444094674692264839657806717923679964139026027126843250459193580680378532734811066291292812871757799957304705959823815337981327892635955884030903952336887398337297071583616148434804992370339552251852390564653345170638429027879494893640162185555746740335701300364549366895603071509335613921022261603757192510216234788297479760745379738926413979105343703865752975290088443822457654233409320835472727119547071827341164598923414643808061642631363409867078064680443778148739652500248351753313127151296891266727601832993148940666076860261312220053449633381926994036057594926536127495919321642620459160643015776389402713240508516646662680872461180106882109916035575188062772680052927761123218010745929633559642537822589267048893038166127679828308107412347151490228081454708475263207431307255929660116229215391718713219296275124965940942264969570604653425158112995766190086742617428331894779561739932248083571327954675626661081416268502138135670526036111028909912905744513739401815865204789074622337967780543030554997349904149174852866314699055775118484307561617715476510223659879006262178277147936319561685416795550343684806292016045430825061295407629630792859378607472703865601637837921366222165768712671192582993050095992902162837409809122249239956018815349224923708459999765889318817603772647160606763515881139107465176471348966415351212897710760490417256912083079981901637762524243714446889324828803989120383830509059162495831234814783149160981763831732072833038894324971545710710008190804215644383909578819102551619474478428657329954383360916910313614352813645944159833591281540014560964897679481295864531058943314040449609060218951113512840094365270382099545227223915145586614988750802750214742576283191925349471448042434102846941962540297936541026820540718958339292143292163938823116188995239202472005635489764201396689638726576661838870926041565986890988555520913227915204944750041326290847931254424080074440665648486008496056685202787762244974149930506977416537062451368978630408639347272514083822569176408650111018195601799779042880484798985309669955587636835226642047780307652329036427222151220573734269706781154856479330513977108408939823425761197980492009203323038357441559071737728375059299834300990403980785087317309678237211414162695404774007206747165603152844307832091465282117853207409348523911168821992539322430533382367874753882421542452546543359486619420823793849027339590189927266342792439160948104213822218739216967292254361917847630601829755876830238862791259734138784172222550622543426119387383737740104020752142091281534932082362029481047655308724606308020905611082998746610739035722505549305859489094152573993473174448601713205053780138594345648815804327800554735984643409551564986962583884769310978837582786634509426107165400044691862785889096328454371614934761199944789347530960557945203896581757968231619206094854804286129511369644900152840416559920260228208747103750562551394236353982547764445613557926830701511359842450458790462806645051722115457122019565954935943198118047568582947950993335515723984720670660910463457195736063977715068710151969448833507033295741927889238677457350474957936970223139373198455841672210106870454983019760834261849916725438709284918960844415873798743398319390999053115880796908621725452782507238244596247743507550993723297438153421700713794146262460973504583532383630067979795827285166067875511097648130873061024949135163260509974603041594613882209798106696373295028893358906861988882106803070819126364653829837094143594360953758425240459676193943231858434420091544108278829125823050127216540862561291333815815917891882114933021145573952982858688405512909936162453690326347437755409384372285222522920728180045196002467872271913615004997124645275778297335044088873243025860723081818176240733540275965692542506748311577974240949543425561130152634195831681482266310865329846404794228984099887003817431935952514958013390643244554627487560307898156274617073857788657582135184601159321290298136722212914863282521260925309643762584662601840201819807347006690730305999309634680059577898115496126040679427026812123499278047987428302883924674410228668173033915416437624288673053432461493434502495570269602833977336669130858166071043116008590160705000907740494697228003075793173013469713164690705004758922728278057227813079498186314023595039759358933422810450331341929311397713658207126691717677878335691177740434601522409130473486822737744612114741932888661749818185189238420974479331886319996130269614670102707039743250659139394447063467707469173685051966719041317024841955178272005799125504626971165859221278224140789326815037008758752489832282928439129560734950215147836308464173977016493289431405436356113395726917126954629133780126458384180129125870730631790722213719524471650861933345488700197431584495034291164118183566718990705708618001597451435020605249210871153129469121553418665308218746303551977820550577365886826535546921752431916056854281227303128784141611869413342368941925157667019406639000655443698705062327976454674007908325531797777200170920718038006559703695480917150127331711148972005103941187410934062470011960606382899852996276308842931818932722548008934316220166413508243656096974843774364568122483054895698919657980527085047017955240561142364808715073571082857799779858575487359739043691565263502161751494179198153302417393450562555972715672624886656879054452611236741488283285013164387141911737214926502144529674841831621740179150134058517779580803184992696478946162737459195526851953673941297538659611911379347500132340340702740471194876444526320453452499856139742662560094705035086488653325646359022974174953413628200886706671043969336656317225914181364868901633383304078875933012989250802781871969334836594758989348182684951736528693850473710969158026082556353263307451145608657688600685809112177946671294246466342459457945913111066747879136806639609689535989475840818938019636996378169519096854574144097135750709680902398937265001096189703148628075292051950637001754977572533951047677728735147314344923802664660278417992389976658773139193698611163219207643945926105993312332154666451751609619860448787473451627755128597988133402844770754951602788283926436193227405030272703073350738663368466123231024735066413185378009697506432103046711726928784833183015418127803485098809834060658656672445633190371507424186454710082217830348745096264952263314055502828054921481164782000783564074473076570693407037183180543231132840365997133898047395013754892577914081407533813881172679102848671760495807653540811325430339562901811492777947612236375451372145148164960497377110680468130432464875760730990473465003392827815368854434033954452224275723629236439045497646649994081824885941319834251627067327245515012422953757567382203061183863112361111893890291919637355110220529675770919406083597586574046920022845441780737467078746067147877540404388982509125620206510518718707762495857617482714825638435028127602077178815441078582045963184452018853571900407232257252004829221628395104072094286315512125866132012851977810429320771571957299454195194595557915818810860815675859338124192816468894096999189626512638162888567663637203369045582388563300461049907279460961646711271724991937034945592240898477673655851148730841113004356785562754125744146775609740791503840899731340008588427699285386009807593832369898883352714841952100958880326076506326879504734739836326158733380822821460104359732672281040247195637417671381461805214241292475142582270469930897253108183290122645884570608221962423855169471267184539959469062613238256338061885654017636466028285028211176436508284000057603667518410702749449032836582852514803909637360163501138858506984897340750715109637047424344170143235428737201550738213842425306603074391112565729451315888550109113853794403072998784154700761828962405192936610674055342279393193931162489283021130993435904552079317731195711479474049063278927305922538253391280088530095689942541814315913098688630969472598984885361849926286108283841804403821798821352441355187675591787478055955886250722870076339490021562692799999052874997412404247312058980214677096061133602291142979890586512965083997824185290295771755155762423675208778327485065146662423891997290725994142719803206390552938641959054670221845602104808500585585289307443162950135568700466953728405562584361183060033978531052771761066380691086850073298216242477319171400120183774473485697544593101159398867818882639914541884690394356014603014659066486109218377985728939371843198308178326299996092450806502890673857403089963766135712693282730463675931698492416300423282058723359156644631333448448340697223426696872258881721523971382531615016457465305710384454752809720361813581207995616373568384574167782652752204951223841342035061501761824165337938683057553650929224610772007851180863780168450083085530554934256124770481106777531434919545708215770992308891501008788817510257566768452278635638544856844862248010112226721524244991390564535578276005679974889418963597452669612078826563311402943533566958418491993579987624057143899026279343725718373806699560661821155986866363530584248797302576369248401744121093742220319052573124028368779737881252195384367600575029531550132019526720645425648230264982371404783207345026262205289867814357240255204102309201009094822061878286019588820059435226845337338338195731746136511045319677167974752732979981394330493469051275770643324396708641560222504670346537379414134244846440426320846188139732750026803337204643522927109212984412900419202866973133102737960830175840977881186118617708232780549455505864313612778249906509228319904758017922651076395740925005069280909637032790900194546397787135618577054971387292946717935268351101241229106868710026586692336371910643239066995087944618601646217219772692349313491925504739911663910835045436664769333263746473557686203872849274715402355003475306958405789247876579931147643818560151607141775233512706365389445209562354206260586029901898056368414866986292207671056885513784467667755153635810970280782474158549412571774780685806536764141420896307498472627732115881433828624027136465381023887243657300005521316506711271628339699480933658439575690433929469186584649834093030241102592314188299333456813647915611320190024202255158614395996799051651051003488873026208178811390683166558800965786851984241023209089025095929241695958402756728784442829224515101146173301504775789699100714539899234834542547693210649649440446431733311057469665064126385864309965973900944926599516608912173391096434371614135658668864080368656056329520099021194384100028294934693258820251431900108339816862173872817244576227658695298218690199894396060581171913595500992568483053761747329190203864367504040818300212993713981312061175816344911838375469350226635823857449879218884337417555239502220561427011462879075545434783190760463150361855392828561877842600863632024683289546426805997798571888151961089455987902060256305703943736784681076578348879473869131473498102955029482868267125039707136405914154768111955242891908706226165779043441390518661993410404715075409585226522274510139034738124716753149372244374139848826974994131564040488707217377783356376906545215460124459915659465635418948823574242955704101556175915573294875082106690280824254219553453194353586269031861592427226655608752537062091711086666662341296958768058171514692870794518706245258471776885591363596539274130280565543503797633003703166917452385287785346328685521326584796327450677640921847544970296403938491437935868400248896166385142627506491122191846428712329475154571787093589194983775973425762588490515207989211595754456962961525871108894632589240812211422645065115401511058885452531063569042929991619303340341981216704213056227399081764218538830752577513041528894826989994892953424269544262252697052432782335811493025871623462472955650679221376534622279455279706311481682293454980405191115348124512457028690914919674646533250848344867559505336778537023761816558993643526349569012334529159388877064422774208935342017743547955353528209170352922322120019250745267134381434742189097874320186156945106492082777020364322660889498947022865152519709016283399182824072682148524860914423997266954323791902293327962576537220555797400215343633771628428021785572835934035541822056893239692874978244022026912095803580274378894627399877979192100531293387881193919122670825844500753204106954048280736983951012160845775591384078721443665892058456059272282764951208859714847525880595921047021697353976869159422819759654015687986622259791340366503397550292522727610696406302742388305180221370322521220164810429339928397965133219292460688138648558372049319005032635009800358093328668306246686374456876793518988238953783423189596805987163077643892428514572504640162276019253105291415150986346170083574901212283860453553631841116428677110820207900300505718178409680061445597729468761462521349305896781068093311498328182146740248240268660808104272177923766912732504746219681968761883102818834144623617660183795210537762050511314928250674890301812928271589291263957955218213095253391873881245054458366742159300431727984262032882553308658376363171116339435462184874574886245836173145412480792109961196353768781899477226446478207683775810089924501629762596246589485842402424748914562117338189287460171190798590653729613898143754724123362193121821908611423086284670776701172546196314721692096247608046857198776245852472382030631422717540199264621661629602664586239675077744067384005818604114602909954316247960549125063361523714860658947041331219566726627023579298379377420965861461109650225534722353903346552912545767356921857793022713892829621949730908910953422162040030269801512664508118441790334558712385077698631557181945102827722951112157667797329516361280640593852025015174678410391593827403824623602587757546523223849244375580463902167915498142187250248435439116329664401474294342072884489971226027652844439435744862477798911445908752336295459190943326166314583647034842774606012966598250341487924352831859856990825050386225745145522535027188870386808569465137865946575449999213767375410422858671687510118379181750396854070748913306849552129797048336503819565045083401446173369389819624286925099427793752430354473033057440033816140357839628318527116255618510633333998819867597317725950712838120802265854192882721448892612176064615232362858476633919584065329801042971941938007318761212229340124827241344176813304763328552271409655417492858124057118699750510292446735765795564976365217164195347712663470236452310702144583451434361781743486412568471115168854801215599668306964593453148111585219392335998138313134600021897277082429826991106113429474377607360347949573357276669998959985924192698632429420971177922942423442358272765382763057378511731151903417914098188130969452955607523877703756471174901064691100052011598083472757203543235758262496686387611218474923923165654713577987155885134103630022219674876336037442240959619837402379886585618014652319786128768404542866765719876636115109142428227951474496441985829009536055319232514540072236249405030280186355982420161660404651470986504999636496495410529945923158193176563933620706573122136613514595713377568928994961821877537091502853637382647867596889775981987270320916758647051524998598395438116716225127088729114515573996281429822261366420555798563716047088609365339919152111561686778921446276104969887483937242793821073096271820269516665622433541530107278032710236056366808288668776199540608947172652824486017055980662911415962118089931962362042336464107622064045083479400645362874946983488314725755936646983096568088651038566734780887003114139266557641530560191194151509753141412524711014434422823723105529063648169458305938276389413928506041813601867683616920681916200502126081562813801207780349357663770921323259690669709004620767619437607768486329180953612159051115559182466361138477090159199699856443961477461148704244598382442514021210236738525480355281238464358183003844441485439854688637295729883954485731815164801928790767190091606038924139325843357451777540141578833887262904163270557116823529776792925396348198873428209523530962273155333837310200394356066024559259673838308289831751156047972298022320618750852473305676430062324258297283358920626458876632563656869090133831776915816387631040847534628063622194189605388242563393775563232239116402575693577978073262451856970066901283979952351938727744751356958866230144004104679739177507693944129226413146306003755711482643826099215953563217027346740129439024773231341152172387047790899515011162416048619925134765284270487812208315159466727018845745237244220117679977443938523884610382384380336548282556241090134418956829267606761314149677571239580662386518988108574182666370875286869780958815403991125347008692655477982801863532855438304737904194848272669969578633615551471501549615472411097391559210093133894803708188851116617415420730454477451956568659857962282116666148182987035845068216643552464723918098563719937585615551156139606957810916440617498547752296239077096166183693599598808572300578097960597821219315343373084063573235960444374798636438213334181591873311551025177272045072867825680311674749051897617899821461418749369333621137114583933680106046401271714440890168441735027176958204970234547028610269506310756811844828313310184266157950178887231070233567097250223166265286166347023231463248982874320074649899495880671762849495283036165636848830994143622255789184784594070456469282817769764839066373418110399375816877995189298473869844729741266063320168869980596229458247887218696090268324791009667313801609060734364769074845780577024193200433427560946524515147560195471004876612141393398526437826643769937429970823517322643896752147367060561748855613294130411511619994479892670500644721650560931006430129737535825833973522436855378577805454152567375343179691617238371188860253122955832257913572721337757907999151004757711367623678425831237259455549070865843892126371174923370998260175002779697303714886663729917572406679417447603983416035057686508983527128676976615632054552575624116681852008314561766749072967529611123205822403757110511534853445302606836913084815545674679193376133150056756342007702381

Both the original decoder and the fast decoder use the same ||(\alpha||), which has 543 digits. Now let's run the decoders.

The original decoder runs in 35.6 seconds and the fast decoder in 2.7 seconds. This is a 13x speedup on my Mac M1 Pro! Since we encode 5 puzzles, each with 900 samples, we've decoded a total of ||(5 \cdot 900=4500||) individual puzzles. Due to adaptive precision, the fast decoder may produce slightly different values after the first ||(p||) bits compared to the regular decoder. We verify that both decoders remain within the theoretical error tolerance of each other.

This is great! We now have a fast decoder implementation that can handle multiple puzzles!

Final Implementation

We are now ready to create the final implementation of the one-parameter model. (Again, because multiprocessing.Pool fails in notebooks we define OneParameterModel in another file and import it here.)

class OneParameterModel:
    def __init__(self, precision:int=8, n_workers:int=8):
        self.precision = precision # number of bits per sample
        self.n_workers = n_workers
        self.scaler = MinMaxScaler()

    @Timing("fit: ")
    def fit(self, X:np.ndarray, y:np.ndarray|None=None):
        # if the dataset is unsupervised, treat X like the y
        if y is None: y = X

        # store shape/size of a single label y
        self.y_shape = y.shape[1:] # pylint: disable=attribute-defined-outside-init
        self.y_size = np.array(self.y_shape, dtype=int).prod().item() # pylint: disable=attribute-defined-outside-init

        # scale labels to be in [0, 1]
        y_scaled = self.scaler.fit_transform(y.flatten())
        assert 0 <= y_scaled.min() <= y_scaled.max() <= 1, f"y_scaled must be in [0, 1] but got [{y_scaled.min()}, {y_scaled.max()}]"

        # compute alpha with arbitrary floating-point precision
        self.full_precision = y.size * self.precision # pylint: disable=attribute-defined-outside-init
        self.alpha = logistic_encoder(y_scaled, self.precision, self.full_precision) # pylint: disable=attribute-defined-outside-init
        return self

    @Timing("predict: ")
    def predict(self, idxs:np.ndarray, fast:bool=True):
        full_idxs = (np.tile(np.arange(self.y_size), (len(idxs), 1)) + idxs[:, None] * self.y_size).flatten().tolist()

        # choose the fast or slow logistic decoder
        if fast: decoder = functools.partial(logistic_decoder_fast, Arcsin(Sqrt(self.alpha)), self.precision)
        else: decoder = functools.partial(logistic_decoder, self.alpha, self.full_precision, self.precision)

        # run the decoder sequentially/in parallel and then unscale+reshape y_pred
        if self.n_workers == 0:
            y_pred = np.array([decoder(idx) for idx in tqdm(full_idxs)])
        else:
            with multiprocessing.Pool(self.n_workers) as p:
                y_pred = np.array(list(tqdm(p.imap(decoder, full_idxs), total=len(full_idxs), desc="Decoding")))
        return self.scaler.inverse_transform(y_pred).reshape((-1, *self.y_shape))

    @Timing("verify: ")
    def verify(self, y: np.ndarray, y_pred: np.ndarray):
        # check logistic decode error is within theoretical bounds (section 2.5 https://arxiv.org/pdf/1904.12320)
        # |y - y_pred| <= π / 2^(p-1) when y, y_pred ∈ [0, 1]
        # |y - y_pred| <= (π / 2^(p-1)) * range when y, y_pred ∈ [min, max], range = max - min

        # multiply the tolerance by the scaler range to account for scaling
        tolerance = np.pi / 2 ** (self.precision - 1) * self.scaler.range
        np.testing.assert_allclose(y_pred, y, atol=tolerance, rtol=0)

The code is quite simple and looks like a standard scikit-learn ML model:

model.fit runs the encoder. It also scales and reshapes the data.
model.predict runs the (fast) decoder. It runs the decoder in parallel and reverses the data scaling, reshaping.
model.verfiy checks that the outputted predictions are within the theoretical error bounds we derived.

The one-parameter model is quite elegant. OneParameterModel itself is ~50 lines of code and the math functions it uses are probably another ~50 lines of code (phi, phi_inverse, decimal_to_binary, binary_to_decimal, logistic_encoder, and logistic_decoder). Only around 100 lines to get a perfect score on ARC-AGI-2!

Conclusion

"With four parameters I can fit an elephant, and with five I can make him wiggle his trunk." - John von Neumann on overfitting

We've built a beautiful model

||[ f_{\alpha, p}(i) = \sin^2 \Big( 2^{i p} \arcsin(\sqrt{\alpha}) \Big) ||]

that achieves 100% on the public eval set of ARC-AGI-2 with one parameter. Although the mathematics is quite nice, this model takes overfitting to its absurd conclusion and directly encodes the entire public eval set of ARC-AGI-2 into the single trainable parameter ||(\alpha||).

Other uses for the one-parameter model

Beyond ARC-AGI-2, the one-parameter model can be applied to all sorts of datasets. For instance, we can encode animal shapes with different values of ||(\alpha||), including the elephant John von Neumann mentioned:

Encode animals with different values of alpha. Figure 1 of 'Real numbers, data science and chaos: How to fit any dataset with a single parameter'.

We can find an ||(\alpha||) that perfectly predicts the fluctuations of the S&P 500 for ~6 months with

alpha = 0.9186525008673170697061215177743819472103574383504939864690954692792184358812098296063847317394708021665491910117472119056871470143410398692872752461892785029829514157709738923288994766865216570536672099485574178884250989741343121

Predict the S&P 500 with 100% accuracy until mid Febuary 2019. From Figure 9 of 'Real numbers, data science and chaos: How to fit any dataset with a single parameter'.

And we can even find values of ||(\alpha||) that generate parts of the famous CIFAR-10 dataset

Encode samples that look like they are from cifar-10. From Figure 3 of 'Real numbers, data science and chaos: How to fit any dataset with a single parameter'.

This technique is incredibly verstile, able to achieve seemingly perfect acuracy across tons of different domains.

To the critics

Yet at the same time, the one-parameter model is incredibly brittle. Simply shuffling the dataset will cause your model to break down as the decoder depends on the index ||(i||), not the sample ||(x_i||). It should be abundantly clear that the one-parameter model has no ability to generalize whatsoever. It would get a 0% on the private, heldout test set of ARCI-AGI-2. Two quick technical notes to the critics. For the complexity theorists, yes, this is cheating. We've violated the fundamental assumption of bounded-precision arithmetic. Most complexity problems assume we operate on a machine with an ||(\omega||)-bit word-size. However, my one-parameter model assumes we can operate on a machine with infinite bit word-size. For the deep learning theorists, of course our one-parameter model can memorize any dataset. Our decoder contains ||(\sin||) which has an infinite VC dimension, i.e. an unbounded hypothesis class, and is therefore infinitely expressive. It can learn anything. What is interesting about the one-parameter model is that it offers a tangible construction, not merely a claim of existence, for learning any dataset.

Lessons

A couple of key takeaways here. Intelligence is not parameter count. The existence of such a simple equation with such powerful expressivity deomonstrates that model complexity cannot be determined by counting parameters alone. The one-parameter model exploits a often-overlooked fact: a single real-valued parameter can encode an unbounded amount of information by hiding complexity in its digits rather than in parameter count. In other words, don't automatically assume that a bigger model is a smarter model. Parameter count can be a poor proxy for intelligence. Intelligence is compression. To compress data, you must find regularities in it and finding regularities fundamentally requires intelligent pattern matching. If intelligence is compression, then our one-parameter model has all the intelligence of a phonebook. It achieves zero compression and is just a nice lookup table. It cannot discover patterns or extract structure. The one-parameter model simply stores the raw data and uses the precision ||(p||) as a tunable recovery knob. Real compression requires understanding. If you want to measure the complexity and expressivity of machine learning models, measure their compression. Use minimum description length or Kolmogorov complexity. These techniques capture whether a model has actually learned the underlying patterns. They cut through the illusion of parameter counts and reveal what the model truly understands. Prof. Albert Gu's paper ARC-AGI without pretraining actually does this right. They used a general-purpose compression algorithm to solve ARC-AGI without training on the test set. That's the real deal. Our one-parameter model is a degenerate version of the same idea. Training on Test Our one-parameter model takes the idea of "training on test" to the extreme: it encodes the entire test set directly into ||(\alpha||), achieving 100% accuracy while learning nothing. The one-parameter model is utterly impractical and, frankly, an absurd hack. But that's precisely the point: it is absurd to train on the test set just to get to the top of a leaderboard. Yet this is exactly what occurs in the AI community. Top AI labs quietly train on their test sets. The one-parameter model does it proudly. It is rumoured these labs have entire teams who generate synthetic dataset for the sole purpose of succeeding on a specific benchmark. This is overfitting taken to the extreme. The ARC-AGI Benchmark ARC-AGI was intentionally designed to resist overfitting. It uses a private test set for official scoring, making training on test impossible. (Our one-parameter model only trained on the public eval set.) Yet even ARC-AGI's organizers have raised concerns about test contamination creeping into modern AI development.

Mike Knoop on ARC-AGI overfitting. Retreived from https://arcprize.org/blog/arc-prize-2025-results-analysis on December 9th, 2025.

Yet there is a more fundamental point: many of the amazing approaches to the ARC-AGI competition seem to overfit to ARC-AGI itself. Researchers generate synthetic data specific to ARC-AGI and create abstractions unique to the grid structure of the competition. I doubht many of their techniques would generalize to other reasoning problems. How many of the innovative solutions to ARC-AGI have inspired downstream improvements in LLMs or other modes of intelligence? I hope these techniques prove to be good for more than just ARC-AGI's delightful puzzles and drive broader innovation the field of AI. Final Thoughts To close, I’ll return to the line we began with:

"When a measure becomes a target, it ceases to be a good measure" - Charles Goodhart

Indeed, the one-parameter model serves as a reductio ad absurdum of what happens when metrics become the goal rather than the guide.

If you liked this or want to chat, reach out! I enjoy talking to people working on interesting problems. Eitan Turok - [Website, X, LinkedIn, Github] To cite this blog post

@online{Turok2025ARCAGI,
        author = {Ethan Turok},
        title = {A one-parameter model that gets 100% on ARC-AGI-2},
        year = {2025},
        url = {https://eitanturok.github.io/one-parameter-model/},
}

Iterations	Decimal	Binary	Observation
0	\|\|(a = 0.431\|\|)	\|\|(\text{bin}(a) = 0.011011...\|\|)	Original number
1	\|\|(D^1(a) = 0.862\|\|)	\|\|(\text{bin}(D^1(a)) = 0.11011...\|\|)	First bit of \|\|(a\|\|) \|\|((0)\|\|) removed
2	\|\|(D^2(a) = 0.724\|\|)	\|\|(\text{bin}(D^2(a)) = 0.1011...\|\|)	First two bits of \|\|(a\|\|) \|\|((01)\|\|) removed
3	\|\|(D^3(a) = 0.448\|\|)	\|\|(\text{bin}(D^3(a)) = 0.011...\|\|)	First three bits of \|\|(a\|\|) \|\|((011)\|\|) removed
4	\|\|(D^4(a) = 0.897\|\|)	\|\|(\text{bin}(D^4(a)) = 0.11...\|\|)	First four bits of \|\|(a\|\|) \|\|((0110)\|\|) removed
5	\|\|(D^5(a) = 0.792\|\|)	\|\|(\text{bin}(D^5(a)) = 0.1...\|\|)	First four bits of \|\|(a\|\|) \|\|((01101)\|\|) removed

Iteration \|\|(i\|\|)	\|\|(ip\|\|) bits removed	\|\|(\mathcal{D}^{ip}(\alpha)\|\|) in decimal	\|\|(\mathcal{D}^{ip}(\alpha)\|\|) in binary	\|\|(b_i\|\|), the first \|\|(p\|\|) bits of \|\|(\mathcal{D}^{ip}(\alpha)\|\|) in binary	\|\|(\tilde{x}_i\|\|), the first \|\|(p\|\|) bits of \|\|(\mathcal{D}^{ip}(\alpha)\|\|) in decimal
\|\|(0\|\|)	\|\|(0 \cdot 6 = 0\|\|)	\|\|(\alpha = 0.50522994995117188\|\|)	\|\|(\text{bin}(\alpha) = 0.\underbrace{100000}_{b_0}\underbrace{010101}_{b_1}\underbrace{011011}_{b_2}\|\|)	\|\|(b_0 = 010101\|\|)	\|\|(\tilde{x}_0 = 0.500000\|\|)
\|\|(1\|\|)	\|\|(1 \cdot 6 = 6\|\|)	\|\|(\mathcal{D}^6(\alpha) = 0.33471679687500000\|\|)	\|\|(\text{bin}(D^6(\alpha)) = 0.\underbrace{\hspace{1cm}}_{b_0}\underbrace{010101}_{b_1}\underbrace{011011}_{b_2}\|\|)	\|\|(b_1 = 010101\|\|)	\|\|(\tilde{x}_1 = 0.328125\|\|)
\|\|(2\|\|)	\|\|(2 \cdot 6 = 12\|\|)	\|\|(\mathcal{D}^{12}(\alpha) = 0.42187500000000000\|\|)	\|\|(\text{bin}(D^{12}(\alpha)) = 0.\underbrace{\hspace{1cm}}_{b0}\underbrace{\hspace{1cm}}_{b1}\underbrace{011011}_{b_2}\|\|)	\|\|(b_2 = 011011\|\|)	\|\|(\tilde{x}_2 = 0.421875\|\|)